A comprehensive survey on reinforcement-learning-based computation offloading techniques in Edge Computing Systems

Producción CientíficaIn recent years, the number of embedded computing devices connected to the Internet has exponentially increased. At the same time, new applications are becoming more complex and computationally demanding, which can be a problem for devices, especially when they are battery powered. In this context, the concepts of computation offloading and edge computing, which allow applications to be fully or partially offloaded and executed on servers close to the devices in the network, have arisen and received increasing attention. Then, the design of algorithms to make the decision of which applications or tasks should be offloaded, and where to execute them, is crucial. One of the options that has been gaining momentum lately is the use of Reinforcement Learning (RL) and, in particular, Deep Reinforcement Learning (DRL), which enables learning optimal or near-optimal offloading policies adapted to each particular scenario. Although the use of RL techniques to solve the computation offloading problem in edge systems has been covered by some surveys, it has been done in a limited way. For example, some surveys have analysed the use of RL to solve various networking problems, with computation offloading being one of them, but not the primary focus. Other surveys, on the other hand, have reviewed techniques to solve the computation offloading problem, being RL just one of the approaches considered. To the best of our knowledge, this is the first survey that specifically focuses on the use of RL and DRL techniques for computation offloading in edge computing system. We present a comprehensive and detailed survey, where we analyse and classify the research papers in terms of use cases, network and edge computing architectures, objectives, RL algorithms, decision-making approaches, and time-varying characteristics considered in the analysed scenarios. In particular, we include a series of tables to help researchers identify relevant papers based on specific features, and analyse which scenarios and techniques are most frequently considered in the literature. Finally, this survey identifies a number of research challenges, future directions and areas for further study.Consejería de Educación de la Junta de Castilla y León y FEDER (VA231P20)Ministerio de Ciencia e Innovación y Agencia Estatal de Investigación (Proyecto PID2020-112675RB-C42, PID2021-124463OBI00 y RED2018-102585-T, financiados por MCIN/AEI/10.13039/501100011033

Improvements to bluetooth low energy and bluetooth mesh towards the new generation of IoT-based heterogeneous networks

In recent years, the term Internet of Things (IoT) has been gaining momentum, moving from traditional sensor networks in home automation to large networks in different areas, such as smart buildings, smart cities or smart factories. The last of these is having a great impact, with different initiatives arising to facilitate its adaptation, in what is already known as Industry 4.0. The applications of these new trends bring with them new requirements that are challenging the current wireless communication networks. These requirements include transmission reliability (zero fails), total area coverage and sustainability, both in terms of network effciency and network cost. In this context, Bluetooth Low Energy (BLE) technology was developed, maintaining the classic Bluetooth objective of facilitating the connection between devices, but with a minimum power consumption. We opted for research into this particular technology because of the ease with which users can be included in the network, through their smartphone or wearable device, while providing low consumption, ideal for sensor devices. However, after an initial evaluation, we found that the topologies included in the specification did not fulfil the requirements demanded by the latest trends in IoT and Industry 4.0. In order to use BLE technology in these new networks, we proposed a standard compliant mesh network using the broadcast capability of the devices. This network was deployed in a real environment, enabling communication between a wide range of devices, including sensor nodes, tablets, wearable devices and a BLE server. The evaluation carried out showed an excellent percentage of transmissions successfully completed, as well as total coverage of the area. Due to the rising number of applications demanding a new topology and the large number of proposals from both academia and companies, the Bluetooth Special Interest Group (Bluetooth SIG) released a suite of specifications to include mesh topology in BLE. This specification is called Bluetooth mesh, and is built over the lower layers of BLE (the Link Layer and the Physical Layer). Bluetooth mesh uses a controlled flood routing through the broadcast capabilities of the BLE devices. The provisioning procedure stands out in the specification. This procedure enables the devices to receive the network key and other information required to take part in the network in a secure manner. Following its release, we decided to focus on the Bluetooth mesh specification, as well as on the devices which used the early versions of BLE. The Bluetooth SIG advanced that any BLE device capable of sending broadcast messages could be part of a Bluetooth mesh network. However, there were no studies on this, and the available implementations were built on the lower layers of the latest versions of BLE. Therefore, and in order to verify this compatibility, this Doctoral Thesis also includes our implementation of Bluetooth mesh, as well as the provisioning procedure, which is supported by many devices. The evaluation carried out with devices with the first versions of BLE showed their compatibility, although these early versions had more limitations than the later versions. Moreover, after the evaluation of the standard provisioning procedure, a lighter alternative is proposed, specially designed for use on devices with the earlier versions of BLE. Finally, focusing on the Bluetooth mesh specification, it is remarkable for its great power consumption, since the devices need to constantly scan the network to receive the messages, which can be sent at any time. To address this problem, and to enable battery-powered devices to take part in the network, the Bluetooth mesh specification proposes a mechanism called friendship. Thus, one of the devices, called friend node, is constantly scanning the network, receiving and storing the messages sent to the low power node. These messages are sent to the low power node on demand, allowing it to scan the network only at short intervals. However, this friendship mechanism uses data transmissions similar to the stop-and-wait protocol, which is highly inefficient. For this reason, Burst Transmissions and Listen Before Transmit (BTLBT) technique was proposed, which minimises the consumption of these nodes

NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium

Producción CientíficaMobile technologies have undergone a great leap forward in a few years, and while 5G networks are already being deployed, there are not yet many proven applications that can fully utilize the advantages of this new technology. Connected and autonomous vehicles are a specific and demanding case, particularly in terms of delay and bandwidth requirements, which can leverage not only 5G but also edge computing technologies. Therefore, the development of testbeds to demonstrate future applications is crucial to enable the full deployment of 5G and edge computing possibilities. In this paper, we present a flexible and modular testbed, targeted towards the evaluation of Cooperative, Connected, and Automated Mobility (CCAM) applications, and we demonstrate a use case (using a 4G system) where an autonomous vehicle offloads processing tasks to an edge server which analyzes images, makes routing decisions, and sends guidance commands back to the vehicle, thus proving the possibilities of edge computing and wireless technologies.Consejería de Educación de la Junta de Castilla y León y FEDER (VA231P20 y VA085G19)Ministerio de Ciencia e Innovación y Agencia Estatal de Investigación (Proyecto PID2020-112675RB-C42 financiado por MCIN/AEI/10.13039/501100011033 y RED2018-102585-T)FEDER a través del Programa INTERREG V-A España-Portugal 2014-2020 (0677_DISRUPTIVE_2_E